Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update puppeteer, node (for perf testing), and the setup script which launches tracerbench #1628

Merged
merged 17 commits into from
Oct 16, 2024
Merged

Update puppeteer, node (for perf testing), and the setup script which launches tracerbench #1628

merged 17 commits into from
Oct 16, 2024

Conversation

NullVoxPopuli
Copy link
Contributor

No description provided.

@patricklx
Copy link
Contributor

patricklx commented Oct 15, 2024 •
edited
Loading

@NullVoxPopuli maybe try upgrading node version in perf.yml to 20.18.0 ?
or maybe its just OOM?

patricklx
patricklx reviewed Oct 15, 2024
View reviewed changes
.github/workflows/perf.yml Outdated

- uses: browser-actions/setup-chrome@v1
with:
chrome-version: 128
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think puppeteer manages it's own chrome version.
You might need to downgrade puppeteer instead.
It had an update 4 days ago. Maybe its related

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I now saw that there is also puppeteer-chromium-resolver. According to docs it should be able to detect local chrome. But i think chrome must be installed before doing pnpm install.

@patricklx
Copy link
Contributor

:( maybe try specific pupeteer version 20.0.0

@patricklx
Copy link
Contributor

Mmm, out of ideas..., looking at the logs it's still using latest chrome?

@patricklx
Copy link
Contributor

It has options for chrome version selection https://www.npmjs.com/package/puppeteer-chromium-resolver#option-from-root-packagejson-with-pcr-object

@patricklx
Copy link
Contributor

I found a workaround. I noticed that the error is around GC multi thread handling. not sure why it happens
So, I found the nodejs flag to make it single thread:
in bin/setup-bench.mjs, line 177 replace command with
node --single-threaded-gc ./node_modules/tracerbench/bin/run

patricklx
patricklx reviewed Oct 16, 2024
View reviewed changes
bin/setup-bench.mjs Outdated
@@ -174,7 +174,7 @@ await new Promise((resolve) => {

try {
const output =
await $`./node_modules/.bin/tracerbench compare --regressionThreshold 25 --sampleTimeout 60 --fidelity ${fidelity} --markers ${markers} --controlURL ${CONTROL_URL} --experimentURL ${EXPERIMENT_URL} --report --headless --cpuThrottleRate ${throttleRate}`;
await $`node --single-threaded-gc ./node_modules/.bin/tracerbench compare --regressionThreshold 25 --sampleTimeout 60 --fidelity ${fidelity} --markers ${markers} --controlURL ${CONTROL_URL} --experimentURL ${EXPERIMENT_URL} --report --headless --cpuThrottleRate ${throttleRate}`;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

./node_modules/tracerbench/bin/run

@NullVoxPopuli NullVoxPopuli changed the title Check in on perf ci Update puppeteer, node (for perf testing), and the setup script which launches tracerbench Oct 16, 2024
patricklx
patricklx reviewed Oct 16, 2024
View reviewed changes
packages/@glimmer/runtime/index.ts Outdated
@@ -1,4 +1,5 @@
/**
* big change
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't forget to remove your big changes :)

@github-actions GitHub Actions
Copy link

github-actions bot commented Oct 16, 2024 •
edited
Loading

duration phase no difference [-172ms to 140ms]
renderEnd phase no difference [-1ms to 2ms]
render1000Items1End phase no difference [-18ms to 9ms]
clearItems1End phase no difference [-2ms to 5ms]
render1000Items2End phase no difference [-17ms to 11ms]
clearItems2End phase no difference [0ms to 0ms]
render5000Items1End phase no difference [-54ms to 30ms]
clearManyItems1End phase no difference [-4ms to 2ms]
render5000Items2End phase no difference [-40ms to 43ms]
clearManyItems2End phase no difference [-3ms to 3ms]
render1000Items3End phase no difference [-22ms to 24ms]
append1000Items1End phase no difference [-34ms to 41ms]
append1000Items2End phase no difference [-10ms to 16ms]
updateEvery10thItem1End phase no difference [0ms to 2ms]
updateEvery10thItem2End phase no difference [0ms to 1ms]
selectFirstRow1End phase no difference [0ms to 0ms]
selectSecondRow1End phase no difference [0ms to 0ms]
removeFirstRow1End phase no difference [0ms to 16ms]
removeSecondRow1End phase no difference [0ms to 13ms]
swapRows1End phase no difference [-1ms to 1ms]
swapRows2End phase no difference [-1ms to 1ms]
clearItems4End phase no difference [-1ms to 3ms]
paint phase no difference [0ms to 0ms]

[16:28:26] Generating Benchmark Reports [started]
[16:28:35] Generating Benchmark Reports [completed]

Benchmark Reports

JSON: /home/runner/work/glimmer-vm/glimmer-vm/tracerbench-results/compare.json

PDF: /home/runner/work/glimmer-vm/glimmer-vm/tracerbench-results/artifact-1.pdf

HTML: /home/runner/work/glimmer-vm/glimmer-vm/tracerbench-results/artifact-1.html

@NullVoxPopuli NullVoxPopuli merged commit cc46230 into main Oct 16, 2024
6 checks passed
@NullVoxPopuli NullVoxPopuli deleted the check-in-on-perf-ci branch October 16, 2024 15:45
@github-actions github-actions bot mentioned this pull request Oct 16, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@patricklx patricklx patricklx left review comments

Assignees
No one assigned
Labels
internal
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@NullVoxPopuli @patricklx